Finger Gesture Recognition Using Microphone Arrays

Although gestures and movement are a natural, everyday occurrence, it remains to be a complex event to interpret by modern day devices. Research has taken a step forward towards solving this problem by discovering alternative methods in an attempt to capture complex gestures. Usually, we interact with our devices by either physical touch or even voice. As these devices become more sophisticated and smaller in size, we should consider whether we could have these devices potentially do more than just simply react to our touch or voice. In this paper, we consider the possibility of interactive and subtle gesture recognition. Specifically, we consider the ability to detect the gestures by emitting ultrasonic waves through the air using microphones and speakers from a typical mobile device. Through movement, these waves are altered, providing ways to interpret gestures. In this work, we particularly focus on finger tapping gestures. We ultimately envision a device that provides the user with the ability to wear it on their wrist, such as a watch. As the user wears the device, it would emit an ultrasonic wave (when prompted), which is then interpreted by it’s own microphones. This challenge is extended further in the case of finger gestures, where the only possible gesture movements that take place are in a close proximity to each other. In this work, we describe finger gesture recognition using a microphone array. First we try to use the reflection of sound waves by implementing an acoustic pulse radar. After understanding we cannot obtain a fine resolution to detect a subtle gesture, we implement a logistic regression classifier using the features from Doppler shift for each microphone. After testing the classifier on different data sets, we conclude that the classifier can be trained successfully with simply a small number of samples for each person. We also find that a learned classifier from one person, does not generalize well to a different person.